FairLens: Auditing black-box clinical decision support systems

The pervasive application of algorithmic decision-making is raising concerns on the risk unintended bias in AI systems deployed critical settings such as healthcare. detection and mitigation model a very delicate task that should be tackled with care involving domain experts loop. In this paper we introduce FairLens, methodology for discovering explaining biases. We show how tool can audit fictional commercial black-box acting clinical decision support system (DSS). scenario, healthcare facility use FairLens their historical data to discover biases before incorporating it into flow. first stratifies available patient according demographic attributes age, ethnicity, gender insurance; then assesses performance groups highlighting most common misclassifications. Finally, allows expert examine one misclassification interest by which elements affected patients’ history drive error problematic group. validate FairLens’ ability highlight multilabel DSSs introducing multilabel-appropriate metric disparity proving its efficacy against other standard metrics.